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Abstract — Facial expressions give important information 
about emotions of a person. Understanding facial expressions 
accurately is one of the challenging tasks for interpersonal 
relationships. Automatic emotion detection using facial 
expressions recognition is now a main area of interest within 
various fields such as computer science, medicine, and 
psychology. HCI research communities also use automated 
facial expression recognition system for better results. Various 
feature extraction techniques have been developed for 
recognition of expressions from static images as well as real time 
videos. This paper provides a review of research work carried 
out and published in the field of facial expression recognition 
and various techniques used for facial expression recognition. 


automatically extracted the whole image sequence. No face 
crop or alignment is necessary. 



One State for 
Brow and Cheek 


Two State? for Eye 


Two State? for 
Furrows 


Three State? for Lip 


Index Terms — Automated facial expression recognition 
system, Face detection, Emotion detection, and Human 
Computer Interaction(HCI). 

I. Introduction 

Detecting emotion has been an increasingly popular research 
topic in recent year. Recent research is done on emotion 
detection from text. Their applications range from 
advertisement and commercial purposes to medical patient 
behavior analysis. Applying this in a social network setting, it 
can be a powerful tool to gain knowledge about how 
individuals, social circles, communities, or cities feel about 
current events or other such topics. Emotion Recognition with 
the help of images is a growing research field. Emotion from 
images is to detect changes in facial expressions in according 
to an individual’s internal emotion state and intentions. 
Human face is a significant place for detecting emotions, six 
emotions detected from human face. They are Happy, 
Surprise, Anger, Sad, Fear and Neutral. This paper has 
covered some techniques which are used for emotion 
recognition from images. The techniques are discussed below 
in brief. 


II. Facial Feature Extraction 

Contracting the facial muscles produces changes in both the 
direction and magnitude of skin surface displacement, and in 
the appearance of permanent and transient facial features. 
Examples of permanent features are eyes, brow, and any 
furrows that have become permanent with age. Transient 
features include facial lines and furrows that are not present at 
rest. In order to analyze a sequence of images, we assume that 
the first frame is a neutral expression. After initializing the 
templates of the permanent features in the first frame, both 
geometric facial features and Gabor wavelets coefficients are 
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Figure 1. Multi-state models for geometric feature 

extraction 

To detect and track changes of facial components in near 
frontal face images, multi-state models are developed to 
extract the geometric facial features (Fig. 1). A three-state lip 
model describes lip state: open, closed, and tightly closed. 

A two-state model (open or closed) is used for each of the 
eyes. Each brow and cheek has a one-state model. Transient 
facial features, such as nasolabial furrows, have two states: 
present and absent. Given an image sequence, the region of 
the face and approximate location of individual face features 
are detected automatically in the initial frame. The contours of 
the face features and components then are adjusted manually 
in the initial frame. Both permanent(e.g., brows, eyes, lips) 
and transient (lines and furrows) face feature changes are 
automatically detected and tracked in the image sequence. We 
group 15 parameters which describe shape, motion, eye state, 
motion of brow and cheek, and furrows in the upper face. 
These parameters are geometrically normalized to 
compensate for image scale and in-plane head motion based 
two inner corners of the eyes. 

Gabor Wavelets : 



Fig.2 Locations to calculate Gabor coefficients in Upper 

Face 

We use Gabor wavelets to extract the facial appearance 
changes as a set of multi-scale and multi-orientation 
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coefficients. The Gabor filter may be applied to specific 
locations on a face or to the whole face image [4, 5, 9, 17, 20]. 
Following Zhang et al., we use the Gabor filter in a selective 
way, for particular facial locations instead of the whole face 
image. 

The response image of the Gabor filter can be written as a 
correlation of the input image /(x), with the Gabor kernel 
Pk(x) 

a k (*£>) = jj 1 (*)Pfc (x - *£>) dx-(i) 

where the Gabor filter pk(x) can be formulated : 

r,2 -k 2 2 / , z*L 

Vk 00 = — - e 2 

where k is the characteristic wave vector. 

In our implementation, 800 Gabor wavelet coefficients are 
calculated in 20 locations which are automatically defined 
based on the geometric features in the upper face (Figure 2). 
We use 5= n five spatial frequencies with wave numbers ki = 
( tt/2 ,tt/4 ,tt/8 ,tt/1 6, tt/32 ), and 8 orientations from 0 to 77 
differing by 77/8. In general, p/x) is complex. 

In our approach, only the magnitudes are used because they 
vary slowly with the position while the phases are very 
sensitive. Therefore, for each location, we have 40 Gabor 
wavelet coefficients. 


III. Face Classification Algorithm : 

The algorithm can be divided into two broad steps: 
registration of a grid with the face and face classification 
based on feature values extracted at grid points. In this paper, 
facial grids are registered either automatically, using labeled 
elastic graph matching or by manually clicking on points of 
the face which describes basic research on facial expression 
recognition).This paper is concerned with face classification 
after the grid has been registered and the algorithm may be 
adapted for use with other grid registration schemes. Labeled 
elastic graph matching has been described in detail in the 
papers cited and will not be discussed in depth here. 

Images are first transformed using a multiscale, multi 
orientation set of Gabor filters (Fig. 3). The grid is then 
registered with the face. Two types of grid are considered in 
this paper: a rectangular grid and a fiducial grid with nodes 
located at easily identifiable landmarks of the face . The 
amplitude of the complex valued Gabor transform 
coefficients are sampled on the grid and combined into a 
single vector, the labeled graph vector (or LG vector in Fig. 
3). The ensemble of LG vectors from a training set of images 
are subjected to principal components analysis (PCA) to 
reduce the dimensionality of the input space. LG vectors 
project into the lower dimensional PC A space (LG-PCA 
vectors). 


Input vectors in the original LG space may then be analyzed 
using the same LDA to determine their attributes. 
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Fig.3 The Gabor -labelled elastic graph representation of 

an facial image. 

The best performances were obtained using a Gabor wavelet 
representation and independent component analysis. All of 
these systems used a manual step to align each input image 
with a standard face image using the center of the eyes and 
mouth.Gabor wavelets can achieve high sensitivity and 
specificity for emotion-specified expressions (e.g.,happy, 
sad) and single AUs under four conditions/1) Subjects were 
homogeneous either all Japaneseor all Euro-American. (2) 
Head motion was excluded. (3)Face images were aligned and 
cropped to a standard size.(4) Specific-emotion expression or 
single AUs were recognized. 

In multi-culture society, expression recognition must be 
robust to variations of face shape, proportion, and skin color. 
Facial expression typically consists of AU combinations ,that 
often occur together with head motion. AU s can occur either 
singly or in combination. When AU occuring combination 
they may be additive, in which the combination does not 
change the appearance of the constituent AUs, or 
non-additive, in which the appearance of the constituents does 
change. The non-additive AU combinations make recognition 
more difficult. 

Investigation of the AU recognition accuracy of Gabor 
wavelets for both single AUs and AU combinations will be 
done. There are three basic steps namely Face detection, 
Feature Extraction, Classification. 

IV. METHODOLOGY: 

A. Face Detection 

Given an image, detecting the presence of a human face is a 
complex task due to the possible variations of the face. The 
different sizes, angles and poses human face might have 
within the image cause this variation. The emotions which are 
deducible from the human face and different imaging 
conditions such as illumination and occlusions also affect 
facial appereances. The approaches of the past few decades in 
face detection can be classified into four: knowledge-based 



14 


www.ijeas.org 













International Journal of Engineering and Applied Sciences (IJEAS) 

ISSN: 2394-3661, Volume-3, Issue-2, February 2016 


approach, feature invariant approach, template -based 
approach and appearance-based approach. 

B. Facial Feature Extraction 

Contracting the facial muscles produces changes in both the 
direction and magnitude of skin surface displacement ,and in 
the appearance of permanent and transient facial features. 
Examples of permanent features are eyes, brow, and any 
furrows that have become permanent with age. Transient 
features include facial lines and furrows that are not present at 
rest. In order to analyze a sequence of images, we assume that 
the first frame is a neutral expression. After initializing the 
templates of the permanent features in the first frame, both 
geometric facial features and Gabor wavelets coefficients are 
automatically extracted the whole image sequence. No face 
crop or alignment is necessary. 


C. EMOTION CLASSIFICATION 



Happiness $a(te$s Anger Disgust Surprise Fear 
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V. CONCLUSION- 

In this paper the automatic facial expression recognition 
systems and various research challenges are overviewed. 
Basically these systems involve face recognition, feature 
extraction and categorization. Various techniques can be used 
for better recognition rate. Techniques with higher 
recognition rate have greater performance .These approaches 
provide a practical solution to the problem of facial 
expression recognition and can work well in constrained 
environment. Emotion detection using facial expression is a 
universal issue and causes difficulties due to uncertain 
physical and psychological characteristics of emotions that 
are linked to the traits of each person individually. Therefore, 
research in this field will remain under continuous study for 
many years to come because many problems have to be solved 
in order to create an ideal user interface and improved 
recognition of complex emotional states is required. In Case 
of a Dark person with a Bright Background, this System will 
be able to detect face with more accuracy. We will be able to 
detect Side Faced Images from them as it will be able to 
extract face feature. In this system there is no as such 
limitations of using Lips as Neuron to detect emotion. So we 
will also be able to detect some of the mixed emotions. 
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